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ABSTRACT 

Problem-based learning (PBL) is an instructional design approach for promoting student learning, in context -rich 
settings. GlobalEd 2 (GE2) is PBL intervention that combines face-to-face and online environments into a 12-week 
simulation of international negotiations of science advisors on global water resource issues. The GE2 environment is 
described examining the impact it has had on middle school students’ written scientific argumentation during the 
intervention. Analyses using HLM on treatment and comparison groups demonstrated a significant positive impact on the 
written scientific argumentation scores of 1818 middle-grade students from two states with an effect size of 0.257 

(p<.001). 
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1. INTRODUCTION 

It has been broadly contended, that to develop a scientifically literate citizenry who can make local and 
global decisions about science related-topics, that science education needs to be grounded in meaningful 
socio-scientific contexts related to the world in which current students inhabit (Anderson, 2002; Sadler, 
2009). Socio-scientific issues are both complex and ill-structured by nature, often not having a single 
clear-cut solution for all the parties involved. Such contextual issues confront students with situations in 
which they have to engage in formulating their own opinions based on data, their own experiences, attitudes 
and values, and collaborative decision-making. Further, they also require students to communicate these 
issues to other in a clear and precise manner. 

Socio-scientific issues are regarded as real-world problems that afford students the opportunity to 
participate in the negotiation and development of meaning through scientific argumentation (Chinn & 
Malhotra, 2002; Osborne, et al., 2004; Schwarz, et al., 2003). Argumentation includes any dialog that 
addresses “the coordination of evidence and theory to support or refute an explanatory conclusion, model, or 
prediction” (Osborne, et al., 2004, p. 995). Research has demonstrated that when students engage in 
meaningful scientific argumentation, they not only learn to develop valid arguments but also learn science 
concepts associated with the topic while they are arguing (e.g., Osborne, et al, 2001; Jimenez & 
Pereiro-Munos, 2002;Schwarz, et al., 2003; Erduran, et al, 2004). 

Unfortunately, all too often inquiry-based approaches to teaching and learning about science that involve 
socio-scientific issues are not employed within typical classrooms (Chinn & Malhotra, 2002; Driver, et al., 
1996; Taber, 2008; Turner, 2008). This lack of socio-scientific inquiry tasks in science classrooms likely 
results from fact that there has been a national and state shift in the science standards towards scientific 
literacy and related pedagogical reform set forth without commensurate alteration of the curricular space 
devoted to the teaching of science in American schools (Sadler, et al., 2007). Inquiry-based curricula, 
especially programs that immerse learners in active investigations of contemporary issues, can consume 
significant chunks of class time. Additionally, the standardized test-driven culture of today’s American 
educational system, the allocation of limited instructional time and resources, are a major concern for both 
teachers and school administrators (Sadler, et al., 2006). Furthermore, research on science teachers has found 
that they report feeling under-prepared and often lack the efficacy necessary to implement and manage 
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socio-scientific inquiry within their science class (Alozie, et al., 2010; Bartholomew, et al., 2004; Bennett, et 
al., 2005; Levinson, & Turner, 2001). So, while it appears that we know what to do in order to develop a 
scientifically literate citizenry and address dwindling science interest and participation among our students in 
STEM, we are simply not doing it as much as we should or could. 

Rather than compete for the already overburdened curricular space devoted to science instruction, the 
GlobalEd 2 Project (GE2) expands the curricular space afforded to the teaching of science by building upon 
the interdisciplinary nature of social studies classes. As problem-based learning (PBL) researchers have 
illustrated, leveraging interdisciplinary contexts, like social studies, as a venue to engage in real world 
problem solving can have a profound and positive impact by deepening students’ understanding, flexibility in 
application and transfer of knowledge (Jonassen, 2009; Koschmann, et al., 1996; Mergendoller, et al., 2000; 
Strobel & van Barneveld, 2009). Because PBL consists of a presentation of authentic problems as a starting 
point for learning, it can increase student motivation and their integration of knowledge (Bednar, et al., 
1992). When working cooperatively in groups within a PBL environment, students learn how to plan and 
determine what they need to solve problems, pose questions, and decide where they can get these answers as 
they make sense of the world around them (Brown, et al., 2008; Lawless & Brown, 2015). 

There can be no doubt that recent USA policy initiatives across local, state and national levels have 
placed increased pressure on schools to improve student performance in the domains of literacy, mathematics 
and science. PBL researchers have illustrated for decades that leveraging interdisciplinary contexts as a 
venue to engage in real world problem solving can deepen students’ understanding, flexibility in application 
and transfer of knowledge (Bereiter & Scardamalia, 1987; Hayes, 2000; Jonassen, 2009; Koschmann, et al., 
1996). Recognizing this, GE2 was created as an educational multi-team simulation that employs educational 
technologies currently available in most schools to build upon the interdisciplinary nature of social studies as 
an expanded curricular application aimed at increasing instructional time devoted to science and persuasive 
writing in a virtual environment (Hayes, 2000) while simultaneously enhancing the social studies curriculum. 


2. GLOBALED 2 AS A SIMULATION: HOW IT WORKS 

The current GE2 simulation operates within a middle school social studies class, focusing on an international 
science crisis created by the simulation staff. GE2 capitalizes on the interdisciplinary nature of social studies 
in order to expand the curricular space for additional opportunities to learn about science on a global scale 
and the use of educational technology, without sacrificing the curricular goals of the social studies 
curriculum, to address the crisis. It works as a simulation environment in which classrooms of students work 
across teams with the goal of reaching an agreement on a critical global science issue, while representing 
their specific assigned country, over a period of 12 weeks. 

The GE2 simulation is implemented by trained teachers within middle-school social studies classes, 
focusing on a specific international science -based crisis. It works as a 12-week, interactive multiplayer 
simulation environment in which classrooms play the role of a specific country with the goal of reaching an 
agreement on a critical global science issue with at least one other of the country-teams in the simulation. 
The scenario developed for the current simulation focused on Global Water Resources (other science -based 
GE2 scenarios include Global Climate Change; International Food Resources and Alternative Energy 
Sources). There are three phases of the 12-week GE2 Simulation: Research, Interactive and Debriefing. The 
simulation is coordinated across multiple classrooms, as each country-team experiences the activities 
associated with each of the three phases (see Lawless and Brown, 2015 for further descriptions of the GE2 
phases) (see figure 1). 
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Figure 1. The Three Phases of the GlobalEd 2 Simulation 

In the Research Phase, within each classroom-team, classrooms are randomly assigned to a country from 
our list of countries across the globe, representing diversity of region, development, size, and resource 
availability (Note: the USA is not randomly assigned to a class and is discussed further in the section below). 
Once the classes are assigned their countries, they are provided with the problem scenario, breaking into 
smaller issue area groups, and work on concrete analytical tasks related to four broad topical issue areas (i.e., 
Human Rights, International Economics, Global Health and The Environment; see figure 2). Students must 
use their research skills and GE2 provided resources to learn about the history, values and customs of their 
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respective countries, as well as the other countries in the simulation, so that they are prepared to make 
appropriate “in character” posts as their assigned country during the negotiations in the Interactive Phase. 

In the Interactive Phase, interactions between classrooms/countries are facilitated through the use of a 
secure, proprietary web-based platform that enables both asynchronous and synchronous written 
communication. Students are told that their country has to “stay in character” (e.g., remain consistent with the 
policy positions and core national and cultural value systems of their assigned country), while attempting to 
develop comprehensive policy responses to particular problems within the issue areas. The electronic 
interactions (asynchronous and synchronous) among the delegates are moderated by a Simulation Controller 
(Simeon), who is trained and knowledgeable of both international affairs and environmental science. The role 
of Simeon is to support/encourage and moderate the interactions so that delegates stay in character, use 
appropriate diplomatic language, and work towards agreements with other country-teams. During this phase 
there are also interactions within the classroom as delegates from the four issue areas discuss strategies of 
developing an agreement on water resources across the four issue areas. It is important to note that within 
each team the same four issue area groups exist and they must coordinate their negotiations such that one 
issue area group does not ignore the importance of another in trying to reach an agreement with another team. 
This is moderated by Simeon. 


The GlobalEd 2 Environment 



TEACHER 
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college student under the 
direction of the project staff 
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control in the simulation. 


Figure 2. The GlobalEd 2 Simulation Environment 

The USA team consists of two undergraduate students in the role of the USA, and is unknown by the 
teachers or GE2 student delegates. The role of the USA is to present a model for negotiations, through their 
diplomatic messages. Additionally, the USA can initiate international problems (e.g., calling for debt 
repayments, sending military to specific parts of the globe, or major policy changes that may affect 
participating countries differently) when the negotiations are unrealistically shallow or lack complexity. The 
USA enables the GE2 staff and Simeon to serve as a model or be disruptive, depending on the needs of the 
simulation during the Interactive Phase. 

In the third phase. Debriefing, classrooms are led by their teachers within the class and Simeon virtually, 
through a series of activities and discussions to reflect on their learning. Students also think about creating 
opportunities for the transfer of new knowledge and skills to other global issues facing the world today, as 
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well as local science literacy topics. The goal of the debriefing is to bring the metacognitive skills gained 
through the simulation to the surface so that students can explore settings in which these new skills may be 
employed within formal and informal learning contexts. 

In summary, GE2 places a pronounced emphasis on the development of students’ understanding of 
science topics, interest in science pursuits, science self-efficacy, and most importantly, written argumentation 
on science topics. For more specific information about GE2 and how the larger context of the simulation 
operates across the three phases of implementation: Research, Interaction and Debriefing see Lawless and 
Brown (2015). 

The core of GE2 is the problem scenario and the interactions across teams that occurs through a 
web-based system enabling email and real-time conferencing in a secure environment. Classrooms of 
students are assigned their country and provided a briefing on the scientific crisis 4-6 weeks prior to the 
launch of the simulation. Students are given analytical tasks related to broad topical issue areas (human 
rights, economics, environment, health) common to all country-teams, and presented in the scenario. Students 
are instructed that their country must “stay in character” (e.g., remain consistent in policy positions and value 
systems of their assigned country), while attempting to develop responses to problems within the four issue 
areas. The scenario developed for the current simulation reported here, focused on Global Water Resources. 
Students did not know the name, race, sex or location of the students on other teams, only the name of the 
country, issue area and student’s initials; there were 17-18 countries in each of the four simulations that are 
reported here. Country-teams in the simulation reported here spanned two different states and across one 
time zone. 


3. STATEMENT OF THE RESEARCH PROBLEM 

GE2’s extensive use of written communications creates an invaluable venue for students to learn and practice 
written scientific argumentation in a real world context and to a large and authentic audience. Research 
illustrates that both instruction and authentic opportunities to write have been shown in the literature to 
improve writing skill (Pajares, 1996; Midgette, et al., 2008). In addition, with more opportunities to 
experience success in writing there is a greater chance to positively impact their writing self-efficacy. Writing 
self-efficacy has been shown to mediate academic performance in writing within the GE2 simulation (Brown 
& Lawless, 2014). As such, GE2 has the potential to impact the quality of students’ written work negotiating 
in the STEM field within the simulation, and may also have an impact on longer term performance. 

The research question addressed here was, is there a significant increase in the quality of students’ written 
scientific argumentation based on participating in a GE2 simulation? The dependent variable examined here 
was the writing quality score of the students in both the GE2 and NEP classes from the pre-and 
post- assessment of the students’ written argumentation skills. 


4. METHODS 

A total of 1818 middle grade students (from grade 7 & 8) participated in the study conducted during the fall 
of 2014 (in either the GE2 simulation or the comparison group - Normal Educational Practice with no GE2 - 
NEP). The student sample was 50.9% female, and 46.3% male (2.8% missing gender information); with a 
nearly equal split between 7 th and 8 th graders. The students in the sample reported their race/ethnicity as: 
44.8% White, 13.4% Black, 26.3% Latino/a, 5.6% Asian/PI, and 7.7% Other, with the remaining not 
reporting race/ethnicity. A total of 51.8% were from urban settings and the remaining 48.2% were from 
suburban settings. There was a nearly equal split between the GE2 and NEP conditions. 

Prior to the class assignment, each teacher received three days of online professional development (PD) 
on the purpose and implementation of the GE2 curriculum. Furthermore, the teachers also received weekly 
PD support and resource support for their students during the implementation of GE2 in the fall of 2014. 

Each of the 51 teachers provided two “approximately” equivalent classrooms. The comparison class 
received the NEP curriculum that the teacher would provide as part of the school or district curriculum. An 
independent consultant blind to class characteristics and teacher/school identity, randomly assigned 
classrooms to either the GE2 or NEP condition. All students in each class participated in the educational 
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activities, but only those who had both parental permission and student assent participated (IRB) in the data 
collection/research components of the study. In this way, each teacher taught both a treatment (GE2) and 
NEP class. 

In order to obtain a measure of implementation fidelity in each classroom, observers were trained to 
record the teaching activities of each teacher in each condition weekly for the 12-weeks of the simulation 
process. Observers used the GE2 Teacher Observation Protocol to collect data on actual implementation of 
the GE2 curriculum and if there was any contamination, or spread of PBL into the NEP class. Initial results 
demonstrated that while there were different implementation practices across teachers, GE2 was 
implemented as a PBL environment in all the GE2 classrooms. 

All consented/assented students completed the battery of pre-test assessments prior to treatment 
assignment; GE2 classes received their country assignments, as well as the GE2 curriculum and resources. 
Within this battery was an open-ended essay prompt - Is the world running out of fresh water? Agree or 
disagree and defend your position. Students had a total of 30 minutes to complete the essay prompt each 
time (pre- and post-). The GE2 assessment battery was administered in a pre-post format to both the GE2 
and NEP students for comparison purposes. All students completed the battery across multiple classes and 
using paper and pencil, so as not to advantage better resourced students. For the purposes of this discussion, 
only one part of the battery will be presented - Written Argumentation. A rubric developed and field tested 
prior to implementation was applied to the essays of all participating students. Each essay was read and 
scored focusing on Claim - Evidence - Reasoning, adapted from the model discussed by McNeill and 
Krajcik (2008), by two independent raters, who maintained over .89% inter -rater agreement. Discrepancies 
were resolved by a third rater. Raters scored essays blind to the pre-post and group of the student (GE2 or 
NEP). 


5. RESULTS 

After collating the data for IRB compliance and removing students with missing data, a final sample of 1818 
students were subjected to HLM analyses examining the impact of the GE2 intervention compared to their 
NEP counterparts with a total of 5 1 teachers. 

Reporting the results in effect sizes (Cohen’s d) using the standard convention indicated that there was a 
significant effect of the GE2 treatment on the scores of written scientific argumentation (pre- to post) 
compared to the NEP students (d=0.257; p<.001) and further, that there was positive impact for both females 
and males. Further details of the treatment effects are presented in tables la-lc (AIC=6779.6; 
Deviance=6773 .6). 


Table la. GE2 vs. NEP HLM Results Using a Fixed Effects Model 


Parameter 

Estimate 

S.E. 

df 

t 

Sig. 

95% Cl 

Lower 

Upper 

Intercept 

4.544 

.090 

33.251 

50.444 

.000 

4.361 

4.727 

Treatment 

.445 

.102 

46.341 

4.333 

.000 

.238 

.651 

cc_writepre 

.256 

.023 

1712.838 

10.750 

.000 

.210 

.303 

tc_cm_writepre 

.674 

.153 

56.041 

4.384 

.000 

.366 

.982 

gc_tm_writepre 

.461 

.207 

17.631 

2.223 

.040 

.024 

.898 
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Table lb. HLM Estimates of Covariance Parameters for GE2 vs. NEP Using a Random Effects Model 


Parameter 

Estimate 

S.E. 

Wald Z 

Sig. 

95% Cl 

Lower 

Upper 

Residual 


2.347 

.080 

29.255 

.000 

2.195 

2.510 

Intercept [subject = TID] 

.062084 

.070 

.878 

.380 

.006 

.578 

57.790 

CID [subject = TIDj 

.119491 

.053 

2.240 

.025 

.049 

.286 

.297 


Table lc. HLM Estimates of Covariance Parameters for GE2 vs. NEP Using an Unconditional Model 


Parameter 

Estimate 

S.E. 

Wald Z 

Sig. 

95% Cl 

Lower 

Upper 

Residual 


2.502 

.085 

29.270 

.000 

2.340 

2.676062 

Intercept [subject = TID] 

.277523 

.094 

2.923 

.003 

.141 

.542 

.539721 

CID [subject = TID] 

.189243 

.068 

2.770 

.006 

.093 

.383 

.388718 


6. CONCLUSIONS 

The results presented here speak to the potential of PBL simulations such as GE2 to provide a meaningful 
learning context within which students can develop their knowledge of socio-scientific concepts as they 
enhance their skills in writing about scientific topics for an authentic audience. GE2 students showed a 
significant increase in their writing argumentation skills compared to the NEP students while participating in 
a PBL focused social studies class. We were also very pleased to see the positive impact for both females and 
males, as they develop the written argumentation skills on science related topics and no significant 
differences in impact based on race/ethnicity or location setting (urban and suburban) once pre-test scores 
were accounted for. We believe that these outcomes may be related to the ability of students to be 
anonymous in their identity as they negotiate online - a specific design feature of GE2. Further, the large 
audience to which students are writing their international proposal to, in order to reach an agreement, may 
have motivated students to engage in the educational activities. 

The data gathered on this sample of student includes additional pre- and post- scales focusing on science 
knowledge, self-efficacy related to technology, science and writing, as well as the message data shared 
during the interactive phase of the negotiations, enabling the research team to track the quantity and quality 
of messages sent, as well as the date/time and recipient information. Further analyses are underway 
examining the student data across urban and suburban settings, the sex of the student and the teacher 
implementation fidelity. These results will include an examination of sex, race/ethnicity, and school setting, 
as we examine potential differential effects of the GE2 intervention for various types of student groups. 

These results must be interpreted with some caution because only one scenario was used in this study 
(Water Resources). Additional studies have been conducted since these data were analyzed enabling us to 
examine the impact of other scenario topics. Further, optimal treatment duration, technology access and 
utilization, teacher training and support, and distal impact are important foci for additional studies to better 
assess both proximal and distal impacts of PBL environments. 

While we believe we still have much to learn about why this intervention produces these significant and 
positive effects for students, we can see that GE2 may afford students the opportunity to develop further 
scientific literacy through PBL simulations (Brown, Lawless & Boyer, 2015; Jonassen, 2009). Further 
analyses of this data set with the other dimensions of the assessment battery and additional data collection 
will enable a fuller investigation of the curricular implications regarding the utilization of interdisciplinary 
PBL educational simulations approaches, like GE2, in promoting STEM literacy among middle school 
students. 
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